pug-lexer

What is pug-lexer?

The pug-lexer npm package is a lexer for the Pug templating language. It tokenizes Pug source code into a stream of tokens that can be used for further processing, such as parsing or compiling into HTML.

What are pug-lexer's main functionalities?

Tokenizing Pug Source Code

This feature allows you to tokenize Pug source code. The code sample demonstrates how to use the pug-lexer to convert a simple Pug template into a stream of tokens.

const lexer = require('pug-lexer');
const tokens = lexer('p
  | Hello, World!');
console.log(tokens);

Handling Indentation

This feature handles the indentation in Pug templates. The code sample shows how nested elements are tokenized, preserving the hierarchical structure.

const lexer = require('pug-lexer');
const tokens = lexer('div
  p Hello
    span World');
console.log(tokens);

Custom Filters

This feature supports custom filters in Pug templates. The code sample demonstrates how a custom filter like 'markdown' is tokenized.

const lexer = require('pug-lexer');
const tokens = lexer('p: markdown
  **Hello, World!**');
console.log(tokens);

Other packages similar to pug-lexer

pug-parser

The pug-parser package takes the tokens generated by pug-lexer and converts them into an Abstract Syntax Tree (AST). While pug-lexer focuses on tokenizing the source code, pug-parser is responsible for the next step in the compilation process.

pug

The pug package is a high-level package that includes both the lexer and parser, along with other components, to compile Pug templates into HTML. It provides a more comprehensive solution compared to pug-lexer, which is focused solely on tokenization.

htmlparser2

The htmlparser2 package is a fast and forgiving HTML/XML parser. While it is not specific to Pug, it provides similar functionality in terms of tokenizing and parsing HTML/XML content. It is more general-purpose compared to pug-lexer, which is specialized for Pug.

README

pug-lexer

The pug lexer. This module is responsible for taking a string and converting it into an array of tokens.

Installation

npm install pug-lexer

Usage

var lex = require('pug-lexer');

lex(str, options)

Convert Pug string to an array of tokens.

options can contain the following properties:

• filename (string): The name of the Pug file; it is used in error handling if provided.
• plugins (array): An array of plugins, in the order they should be applied.

console.log(JSON.stringify(lex('div(data-foo="bar")', {filename: 'my-file.pug'}), null, '  '))

[
  {
    "type": "tag",
    "line": 1,
    "val": "div",
    "selfClosing": false
  },
  {
    "type": "attrs",
    "line": 1,
    "attrs": [
      {
        "name": "data-foo",
        "val": "\"bar\"",
        "escaped": true
      }
    ]
  },
  {
    "type": "eos",
    "line": 1
  }
]

new lex.Lexer(str, options)

Constructor for a Lexer class. This is not meant to be used directly unless you know what you are doing.

options may contain the following properties:

• filename (string): The name of the Pug file; it is used in error handling if provided.
• interpolated (boolean): if the Lexer is created as a child lexer for inline tag interpolation (e.g. #[p Hello]). Defaults to false.
• startingLine (integer): the real line number of the first line in the input. It is also used for inline tag interpolation. Defaults to 1.
• plugins (array): An array of plugins, in the order they should be applied.

License

MIT